getting and posting http in bash
howto make use of the builtin network capabilities of bash and do complete webscripting without using external binaries.
//requirement// you need a fully compiled bash for this to work. The stripped down version provided by Debian for instance will not work. They configured and compiled their bash with //--disable-net-redirections//. So actively disabling the /dev/tcp/ interface.
these functions use the /dev/tcp interface to get stuff from the web. (interface /dev/tcp: /dev/tcp/<HOSTNAME>/<PORTNAME> this virtual file acts just like a fifo)
**update** WARNING: i replaced these functions with external binaries in most of my own scripts again by using functions like
#web_get "<hostname>" "<portname>" "<urltofile>"
web_get () {
local HOST="$1"; shift
local PORT="$1"; shift
local REQURI="$1"; shift
curl "http://$HOST:$PORT$REQURI" 2> /dev/null
}
the /dev/tcp interface has buffering issues that make it unsuitable for large transfers. There is a limit on input data it can handle from the server, when reached it, //don't remember exactly//, either killed the script (SIGSEV) or misteriously just sat there doing nothing.
anyhoo, for informational purposes, and without further delay;)
# weblib.sh GPL 2004 Thijs Dalhuijsen
# http://thijs.dalhuijsen.com
# SYNOPSIS:
# =========
# web_get "<host>" "<port>" "</properly%20formatted/uri/to.file>"
# web_head "<host>" "<port>" "</properly%20formatted/uri/to.file>"
# web_post "<host>" "<port>" "</properly%20formatted/uri/to.file>" "<post%20data=postdata>"
web_get () {
local HOST="$1"; shift
local PORT="$1"; shift
local REQURI="$1"; shift
exec 5<>"/dev/tcp/${HOST}/${PORT}"
echo -e "GET ${REQURI} HTTP/1.1\nHost: ${HOST}:80\nReferer: http://images.google.com/\n\n" >&5
local IFS="^M"
local DW=0
local x
local RES=
for x in $(cat <&5); do
if (( DW > 0 )); then
(( DW++ < 2 )) || RES=${RES}${x}
fi
if (( ${#x} == 1 )) ; then
(( ++DW ))
fi
done
echo -n "${RES}"
}
web_head () {
local HOST="$1"; shift
local PORT="$1"; shift
local REQURI="$1"; shift
exec 5<>"/dev/tcp/${HOST}/${PORT}"
echo -e "HEAD ${REQURI} HTTP/1.1\nHost: ${HOST}:80\n\n" >&5
echo -n "$(<&5)"
}
web_post () {
local HOST="$1"; shift
local PORT="$1"; shift
local REQURI="$1"; shift
local PAYLOAD="$*"
exec 5<>"/dev/tcp/${HOST}/${PORT}"
echo -e "POST ${REQURI} HTTP/1.1\nHost: ${HOST}:80\nContent-Length: ${#PAYLOAD}\n\n${PAYLOAD}" >&5
local IFS="^M"
DW=0
# FIXME: doesn't &5 expand to evalable now?
for x in $(<&5); do
if (( DW > 0 )); then
echo -n "$x"
fi
if (( ${#x} == 3 )); then
DW=1
fi
done
echo -n "$(<&5)"
}
|